[11] Store work summaries as row logs - #1389

Makes work summary updates cost the same at any size. Every request, response, billing change, edit and commit used to read, merge and rewrite the whole work/<id>/work.json, so large works slowed the TUI and eventually broke /work. Now work.json is a small manifest and the rows go into append-only logs.

Stacked on #1386. Ship it in the same release as #1348 and #1386, or before them: #1348 makes each request row about five times larger, and #1386 adds the /work browser that reads these files.

Problem

• Growing one work to 2,000 requests wrote 17.6 GB to disk.
• Each update blocked the event loop longer as the work grew: 229 ms at 20,000 requests.
• Above about 2,800 billed requests work.json passed the browser's 8 MiB read limit, so /work showed "Summary unavailable" and lost the work's PR state.
• Opening /work with 30 medium works parsed about 190 MB of JSON.

How it works

work/<id>/work.json                      # version 2 manifest: what /work lists, one entry per row log
work/<id>/rows/<collection>.<g>.jsonl    # one row per line, sessions included; a later line with the same key wins
work/<id>/cost-totals.json               # new: the bounded figures /work prints
work/<id>/costs.json                     # unchanged shape, refreshed at most every five minutes

• Writes: under the existing per-work lock, a writer appends only changed rows, syncs them, then replaces work.json, which commits them. Readers stop at each log's committed length, so an interrupted append is ignored and the next writer cuts it off. Merge rules are unchanged.
• Reads during an update: each process remembers where rows are, up to 200,000 rows across works. It reads a whole log once, the first time it updates that work or after another process compacted it; later updates read only the lines they change.
• Compaction: when superseded lines outweigh half a log's live rows and exceed 1 MiB, the latest line per key is copied into a new generation.
• /work reads only manifests and cost-totals.json, never the row logs. The text /work in ACP and print mode uses the same totals.
• costs.json keeps its shape but is rewritten at most once every five minutes per work (immediately on a process's first pass, or when it or cost-totals.json is missing).

Evidence

Measured during development on an Apple M4 Pro, base 74f8e0d0 against an earlier build of this branch, median of 15 updates in one process:

• Growing one work from 0 to 2,000 requests: 17,591 MB written → 19.4 MB.
• Opening /work with 30 works of 5,000 requests: 379 ms, +474 MB heap and 0 of 30 summaries readable → 3 ms, +1 MB heap, 30 of 30.
• Lab on a binary built from this head: one work with 5,000 seeded requests had an 8.3 MiB requests log and a 1.1 KB manifest; live updates took 19, 40 and 40 ms and /work opened in at most 357 ms.

Review fix on 289de5c77 (merged up as ea3985959): the version 1 reader, migration, fixtures and the summary too large row are removed (−1,002 lines). Two new tests fail on the previous head; pnpm check and 1,280 work attribution and reporting tests pass.

12 October retest on a binary of ea3985959 (later commits change one test and the docs): real GLM runs on GitHub and GitLab, 3 uploads each accepted, and /work equal to the backend's totals (PR #929 $0.0217, MR !930 $0.0402). Row logs and cost-totals.json were written. The lab passes 73/73 after its expectations were updated for readable prices and the new status layout. The TUI e2e test that still expected version: 1 in the work summary is fixed in 456c3bf94; CI passes on 51e265b0f.

On 95614e7c6, after the final review fixes:

• CI: build, pr-checks, master-checks, TUI e2e in 4 shards, ACP e2e and MCP e2e pass on this head. A duplicate tui-e2e entry from a superseded workflow run shows as cancelled.
• pnpm check passes; the full unit suite passes except 5 tests that need tools this machine lacks: a DAP js-debug adapter, a built validate-skill.mjs and a binary rebuilt for the smoke test.
• New test: a manifest that still lists its sessions reads unchanged, and the next update moves them into the sessions log.
• Lab on a binary built from this head (scripted model; local GitHub, billing and report fixtures): 73/73 default scenarios pass.

Tests also cover:

• seeded random sequences over all ten record types, through compaction, compared after every batch with the same merge applied to every row in memory (the merge loop itself is unchanged);
• an old version 1 work.json deleted by /work and by the next update, with the work rebuilt from its journals;
• interrupted appends, missing, short or damaged logs, deleted manifests, crashes during compaction, and concurrent readers and writers across processes;
• a restarted process that reads an unchanged log once, not on every update;
• /work cost lines from cost-totals.json equal to the previous output for every saved report in the cost-sync tests, connected works included, and totals that are written again after being deleted.

Compatibility and risks

• Version 1 summaries are not read. A work.json that is not a version 2 manifest is deleted the next time /work lists the work or an update opens it; no other file is touched. The existing journal recovery then writes a version 2 summary, on the work's next update or at the next launch. Until then /work leaves the work out. Cost files, the text /work cost lines and PR cost reporting read the journals and are unaffected.
• Damage: a missing or invalid manifest, a short log or a damaged row rebuilds the work from its journals.
• Downgrade to v1.7.1 or older: the old version rebuilds a version 1 file from the journals; this build deletes it and rebuilds again on return.
• The /work header now adds each listed work's connected cost report once, so connected works that are not listed also count. [10] Browse works, costs and PRs in /work #1386 counted only the listed works' own requests.
• First update after a restart reads the work's logs once; the cost is proportional to the work's size.
• Not changed: a cost pass after a journal change still rereads every journal, and costs.json still repeats a connected group in each member's file.

docs/work-attribution.md lists each row log and its keys.